Multi-Sensory Urban Search-and-Rescue Robotics: Improving the Operator’s Omni-Directional Perception

*Correspondence: Robert W. Lindeman, Human Interaction in Virtual Environments Laboratory, Department of Computer Science, Worcester Polytechnic Institute, 100 Institute Road, Worcester, MA 01609, USA e-mail: [email protected] The area of human–robot interaction deals with problems not only related to robots interacting with human beings but also with problems related to human beings interacting and controlling robots. This article focuses on the latter and evaluates multi-sensory (vision, hearing, touch, and smell) feedback interfaces as a means to improve robot-operator cognition and performance.The paper summarizes three-previously reported empirical studies on multi-sensory feedback using simulated robots. It also reports the results of a new study that used a physical robot to validate the results of these previous abovementioned studies, and evaluate the merits and flaws of a multi-sensory interface as its sensorial complexity was gradually increased.The human senses were selected based on their response time to feedback and easiness of adaptability of their feedback mechanisms to different types of robot-sensed data. The results show that, if well-designed, multi-sensory feedback interfaces can indeed improve the robot-operator data perception and performance. They shed some light on the benefits and challenges multi-sensory feedback interfaces bring, specifically on teleoperated robotics and urban search-and-rescue. It adds to our current understanding of these kinds of interfaces and provides a few insights to assist the continuation of research in the area.